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ABOUT THIS CASE STUDY 


Achieving the Dream (ATD) is one of 12 higher education and digital 
learning organizations that make up the Every Learner Everywhere (Every 
Learnen Network, whose mission is to help higher education institutions 
improve and ensure more equitable student outcomes through advances 
in digital learning, particularly among poverty-impacted, racially 
minoritized, and first-generation students. Every Learner partners are 
addressing high failure rates in foundational courses through the provision 
of scalable, high-quality support to colleges and universities seeking to 
implement adaptive courseware on their campuses. As part of its ongoing 
effort to help community colleges develop effective teaching and learning 
practices, ATD is working with seven community colleges in Florida, Ohio, 
and Texas on this initiative, providing coaching and direct support to the 
colleges, fostering collaboration within and among the participating 
institutions, and serving as a liaison to the Every Learner network. 


The following case study is part of a series of studies conducted by ATD 
examining how adaptive courseware is implemented at those institutions 
as well as how courseware is used in particular disciplines to better serve 
students. Case studies are based on a series of interviews with college 
leaders, faculty, instructional designers, developers, technology specialists 
and students who were enrolled in classes using the Courseware. 
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OVERVIEW 


Faculty in several disciplines at Lorain County 
Community College explored the use of adaptive 
courseware, ultimately implementing it in a gateway 
statistics Course and several business Courses. 


Administrators framed 
participation in the Every 
Learner Network as part of a 
broader education technology 
capacity building effort that 
provided important support 
and time for participating 
faculty members to explore 
and implement adaptive 
courseware. 


Individual faculty members 
evaluated courseware to 
determine whether it met the 
specific needs of their courses, 
ultimately finding aligned 
products in business and 
statistics. 


Statistics faculty intentionally 
changed grading policies to 
encourage student usage of 
adaptive courseware, which 
was part of a broader redesign 
of supports for the gateway 
course with the highest 
enrollments at LCCC. 


Students and faculty both 
stressed the importance of 
faculty support above and 
beyond assignments provided 
through the courseware. 


MB SUPPORTING INSTITUTIONAL REFORM 


The Every Learner initiative supports broader efforts to 
foster student learning with evidence-based practices, 
including efforts to support the development of students’ 
mastery of concepts and critical thinking skills. “The 

use of adaptive courseware in our statistics course 
supports our student-ready approach to teaching and 
learning,” says LCCC President Dr. Marcia Ballinger. The 
initiative also reflects the contexts in which broader 
institutional reform is taking place at community colleges 
throughout the ATID Network, including building a culture 
of excellence in teaching and learning and leveraging 
data and technology to support student success and 
equitable student outcomes. To learn more, see p.7. 
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INTRODUCTION 


Lorain County Community College student Sydney 
Wade hopes to pursue a career in the medical or 
environmental fields. Her first math course at the Ohio 
college was a critical first step. 


“Statistics was a precursor to 

all my future math and science 
courses,” says Wade, who plans 
to transfer to Bowling Green State 
University next fall. 


As Oc prerequisite for majors in allied 
health fields, introductory statistics 

is the math course with the largest 
enrollments at LCCC. Faculty 
leveraged adaptive courseware as 
part of a broader curricular redesign 
to help more students complete 

the gateway course during their first 
year—a particular challenge given 


that statistics is very different from 
the math courses students have 
taken before it. The nature of the 
course also requires faculty to more 
closely evaluate the questions that 
adaptive courseware automatically 
generates to avoid presenting 
students with problems that are 
difficult or impossible To solve. 


“Statistics is just a finicky beast— 
you can’t just fiddle with the 
numbers,” says math faculty 
member lan Morrison. “Problems 
have to be curated.” 
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BEYOND THE NUMBERS 


Historically, LCCC students who placed directly into 
introductory statistics had tended to do well. However, 
the majority of students start in develoomental math 
courses, which focus on prerequisite concepts like 
algebra that don’t directly correspond with the 


concepts taught in statistics. 


Adaptive courseware was 

adopted as part of a “multifaceted 
approach” to redesigning the 
gateway statistics course, says 

Aaron Weiss, dean of science and 
mathematics. First, faculty created 

a paired developmental course that 
students took at the same time as the 
college-level statistics course to build 
prerequisite skills. Within the statistics 
course itself, adaptive courseware 
was used as a way for students to 
practice concepts outside of class 
and to prepare for class discussions. 
That added support has proven to 
be critical, given that most students 
find the concepts behind statistics 
unfamiliar and challenging. 


“Students want to go step 1, 2, 

3. Statistics isn’t that—you have 

to decide when to do what. My 
students think they‘re going to fail 
before | say one word to them,” 

says Kati Dobeck, a math faculty 
member. “The more familiar they are 
when they access the material, the 
more at ease they'll be.” 


Dobeck had experimented with 
adaptive courseware in her courses 
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prior to LCCC’s work with the Every 
Learner Network. Lessons from those 
earlier iterations informed the rollout in 
statistics, including replacing in-class 
quizzes with graded adaptive work 

to provide better feedback to guide 
instruction and ensure that students 
engaged with the courseware. 


“We called adaptive work ‘quizzes’ 
so they understood it had higher 
stakes than regular homework,” 
Dobeck says. “Students don’t do 
optional, so something for a grade is 
really important.” 


Several other faculty members 
who taught statistics participated 
in the initial Every Learner 
implementation, saying they 
found the focus on adaptive work 
beneficial for both the students 
and themselves. “This software 
mixes up and throws it at them, not 
randomly, but in ways that forces 
them to think about concepts 

and come up with study strategies 
and some critical thinking skills,” 
says Morrison. “| like it as an extra 
tool we have to give students 
something to think about.” 


Like other community colleges 
participating in the Every 
Learner grant which are part 
of the ATID Network, LCCC has 
committed to engaging in 
bold, holistic, and sustainable 
institutional change across 
multiple institutional areas 
and priorities. The institution’s 
efforts to implement adaptive 
courseware reflect the 
importance of several key 
cornerstones of institutional 
change, including building 

a culture of excellence in 
teaching and learning and leveraging data and 
technology to support student success and equitable 
student outcomes. “The learning technology helps 
our faculty meet students where they are, creates 
rich opportunities to help students reach their full 
potential, and prepares students with knowledge and 
skills fo thrive in a changing world,” says Dr. Marcia 
Ballinger, the college’s president. 


AID’‘s Institutional Capacity Framework and Institutional 
Change Assessment Tool (ICAT) outlines seven essential 
institutional capacities required to create a student- 
focused culture that promotes student success. One 
focuses specifically on teaching and learning and 

the commitment to engaging full-time and adjunct 
faculty in examinations of pedagogy, meaningful 
professional develooment, and a central role for 
faculty as change agents within the institution. Building 
capacity in this area is particularly crucial because, 

as AID President Dr. Karen A. Stout recently asserted, 
“focusing on teaching and learning is still not central to 
the field’s overall theory of change. We still have much 
more to do to build a deep focus on pedagogy and to 
support our colleges in building a culture of teaching 
and learning excellence.” 


To foster this culture of teaching and learning 
excellence, AID‘s Teaching & Learning Toolkit: A 
Research-Based Guide to Building a Culture of 
Teaching & Learning Excellence is centered on 
four cornerstones of excellence that provide a 
forward-looking vision that campuses can use to 
inform their work. 


Me BUILDING ON AID S CORNERSTONES OF EXCELLENCE 


Initiatives such as Every 
Learner provide important 
resources and supports to 
community colleges and the 
time, space, support, and 
resources to explore innovative 
pedagogical approaches to 
improving student learning 
and outcomes. They also offer 
sustained opportunities to 
build on these cornerstones 

of excellence. LCCC’s work 
with the initiative exemplifies 
the importance of institutional 
efforts to empower faculty to 
consider, adapt, test, and refine new approaches 
to fit their campus context and the needs of their 
students. “It can be very time consuming to do 
research, build and implement, and train others. The 
hardest part is taking the initial steps ... and allowing 
for the appropriate time for the faculty member to 
research and potentially build the tools,” says Aaron 
Weiss, dean of science and mathematics. 


This commitment builds on previous experiments 
with adaptive technology by individual statistics 
and business faculty members, which guided 
broader efforts to integrate and scale the 
technology in ways that aligned with evidence- 
based instructional practices that fostered student 
learning. In statistics, previous experiments informed 
efforts to integrate the technology into a broader 
redesign of the gateway statistics course, while 
adaptive courseware supported efforts to flip the 
classroom model in business courses. “It overall lines 
up with what we do in class, which | think students 
find helpful,” says statistics faculty member Lisa 
Sheppard. Faculty also leveraged analytics from 
student work in adaptive courseware to pinpoint the 
ones who needed extra support. 


The Every Learner grant also provided faculty 
with opportunities to collaborate with the ATD 
Network. “From an institutional perspective, it was 
an opportunity to learn from other institutions and 
get coaching around organizational pieces,” says 
Karla Aleman, former dean of LCCC’s library and 
eLearning. 
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EXPANSIONS AND SETBACKS 


Efforts to implement adaptive courseware saw successes 
and setbacks in other disciplines at LCCC. 


Like Dobeck, business faculty 
member Jerry McFadden had 
experimented with adaptive 
courseware in several of his 
business courses before joining 
the Every Learner project at LCCC. 
He has since rolled out adaptive 
courseware across all of his business 
courses after receiving positive 
feedback from students and 
observing that they were reading 
assigned materials before class. 


“It forces them to do what alll 
students should do—read the 
chapters,” he says. “It essentially 
makes it a prerequisite.” 


McFadden uses courseware as 
part of a flipped classroom model, 
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with students reading materials 
and answering adaptive questions 
ahead of lectures. The questions 
help ensure “they're learning the 
concepts before | discuss them,” 
he says. “That was the light bulb for 
me—it’s helping them in a way they 
couldn't get from me lecturing. We 
all have different learning styles, 
and this is another tool in the toolkit 
they didn’t have before.” 


Business students say that 
adaptive courseware helps 
them grasp concepts that were 
more difficult to hone in lengthy 
textbook assignments. “We 

read material and answer the 
questions, and you retain the 
material—it works,” says business 
student Alichielle Shears, who 
returned to LCCC in fall 2019 after 
stopping out a decade earlier. 


LCCC was one of a relatively 

small number of institutions which 
experimented with adaptive 
courseware in English. Faculty 
hoped to use adaptive courseware 
To support the students with the 
greatest needs in newly redesigned 
developmental courses that had 
been integrated to combine 
reading and writing skills. 


“A lot of the conversation with 
faculty was about students coming 
in with reading levels below 7th 
grade,” says Dr. Brenda Pongracz, 


dean of arts and humanities. 
“Students just couldn’t read at the 
level required to pass the writing 
(requirements). They’re not going to 
be successful in college if they can’t 
read at a college level.” 


To that end, faculty wanted to use 
adaptive courseware fo support 
the development of reading 
comprehension skills to allow 

them to focus more intensively 

on writing in class. The goal, says 
developmental English faculty 
member Patty Mack, was to “isolate 
a few concepts that give a student 


Mack and other faculty members 
explored existing products and had 
conversations with peers at other 
Every Learner institutions in search of a 
solution. While adaptive products exist 
that focus on the fundamentals of 
grammar, they found little that served 
the reading needs of their students. 
Even products targeted at K-12 
students with similar skill levels focused 
on materials that weren‘t compatible 
with college-level reading. 


“Nothing | looked at matched the 
needs,” says Mack. “I’m going fo try 
to teach my section conventionally 


a leg up to success, particularly 
directed at reading.” 


MTHE STUDENT EXPERIENCE 


Many LCCC students enrolled in introductory 
statistics Courses are seeking careers in 
nursing, sO perhaps it’s not surprising that in 
conversations about adaptive courseware 
they often returned to the importance of 
feeling supported and cared for—regardless 
of technology or modality. “The professor is 
very important to how I’m doing in the class,” 
says Samuel Bitter, who returned to Lorain as a 
career changer seeking a degree in nursing. 


Students generally found adaptive practice 
within the courseware helpful, but particularly 
when supported by professors in synchronous 
online settings. Another returning student 
pursuing a career in nursing, Maria Cristarella, 
credits her professor with beginning each 
class session by asking whether students 

had trouble with their assignments in the 
courseware. “He'll go through the entire 
homework assignment if we want him to,” 

she says. “As far as his notes and PowerPoints 
go, he gets some of the questions off the 


this year and keep trying to get it 
right in my class.” 


homework because we've seen them before. 
It makes it a bit easier.” 


Other students stressed the importance of 
faculty support during classes because of 

the limitations courseware can place on their 
ability to help while students are working 
asynchronously. “The adaptive quizzes time 
each question you're working on, so you can’t 
send them to your professor to have them help 
you,” says freshman Melanie Medina, who wants 
to become a school counselor. “She can’t do 
anything about how it’s structured, and she 
can’t see the question I’m seeing.” 


While recognizing the challenges of the shift to 
online learning in 2020, students urged faculty to 
support their work within adaptive courseware 
as much as possible. “Time and caring may 
sound weird when you're taking about statistics, 
but there’s a difference between having 

to teach the material to yourself and being 
taught,” Bitter says. 
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THE IMPACT 


Statistics faculty found initial outcomes promising. Nearly 
as many students completed the gateway course in fall 
2019 as during the entire previous year. Overall student 
grades also tended to mirror their scores in adaptive 
courseware more closely than the previous assignments, 
suggesting the homework was helping them master the 


course material. 


Faculty in both statistics and 
business Courses are now working 

To scale the use of adaptive 
courseware across additional 
sections, through both informal peer 
learning and by creating resources 
for other faculty members. In 
statistics, Dobeck developed a 
master course shell in LCCC’s 
learning management system, with 
the idea that other faculty members 
could adapt it within their sections. 
In similar fashion, McFadden found 
that sharing adaptive resources with 
adjunct business faculty members 
helped encourage some who were 


less familiar with the technology to 
adopt it, particularly after classes 
shifted online during the pandemic. 


In Fall 2020, almost all of the nearly 
30 sections of the introductory 
statistics course, including those 
taught by full-time and adjunct 
faculty members, are using the 
adaptive courseware, giving 

the institution the opportunity 

To measure the impact of the 
technology at scale. “It will be 
interesting to see a larger student 
response across the majority of our 
sections,” Weiss says. 
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What Worked Well: 


Integration. Statistics faculty appreciated that 

the adaptive courseware selected for the pilot 
was developed by the same publisher they were 
using for the course. “We had already adopted 
this textoook and had been using the online 
homework,” says math faculty member Kati 
Dobeck. “They liked the pedagogy of the textbook 
and the product was already there and seemed 
valuable to their learning.” 


Other faculty members said that integration helped 
ensure materials were aligned. “It overall lines up 
with what we do in class, which | think students 

find helpful,” says statistics faculty member Lisa 
Sheppard. “It would be more difficult if things didn’t 
match up with what we're doing in the chapter.” 


Students, including Sydney Wade, generally 
agreed that alignment between the courseware 
and assessments helped them master needed 
concepts. “| would find the questions in homework 
assignments in the courseware were the same 

as on the test,” she says. “I thought | had a pretty 
good grasp of the concepts.” 


Adaptive practice. Faculty and students agreed that 
the premise of adaptive practice was particularly 
well suited for the business and statistics courses that 
used courseware. Students, mathematics faculty 
member Blerta Ereditario says, “like the experience 
of being able to practice over and over again.” 
Business faculty member Jerry McFadden adds that 
the courseware helps students master key concepts 
and assess their progress. “Self-assessing is hard, and 
it reinforces and it’s a nice gauge,” 


Student Alichielle Shears agrees. “It helps a lot,” 
she says. “It structures the homework and the 
studying. | would rather do it online than turn 60 
pages in a textbook.” 


Pacing. Students in statistics class credit the weekly 
graded assignments with helping them stay on 
track. “It makes it easier because it’s what you just 
learned,” says Maria Cristarella. Freshman Melanie 
Medina agrees: “It’s divided up enough so it’s 

not overwhelming,” she says. “If you were trying 
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to figure out when to do everything by yourself, it 
would be more difficult.” 


Data reporting. Faculty members credit courseware 
with providing them insights into which students 

are struggling so they can intervene. “I can look 

up individual students and see where they're 
struggling,” McFadden says. “I’ve been able to 
help a few students that way.” 


Cost. In statistics, the courseware was included 

in textbook costs, which had fallen to below $60. 
“Faculty didn’t want them to pay for another 
thing,” Dobeck says. “Students don’t have a lot of 
money, and we don‘t want them to spend money if 
they‘re not going to have a lot of value.” 


Ongoing Challenges: 


Onboarding. Some students reported that faculty 
didn’t provide a specific introduction to how to 
use the adaptive courseware and why it would 
benefit them. While some said this was a minor 
issue—“it was all pretty easy to figure out once you 
got started,” Wade says—faculty acknowledged 
that an explicit introduction can help explain the 
purpose of adaptive practice. 


“Students sometimes have a hatred of the 
courseware, and often it’s just because they don’t 
understand how it works,” Dobeck says. “They 
didn’t realize they could keep working to improve 
their grade.” To that end, statistics faculty created 
a video illustrating how adaptive assignments 
work as students progress through the m—"from 
beginning to end,” Dobeck says. 


Difficulty. Faculty noted a lack of consistency 
between the difficulty levels of questions in adaptive 
courseware and their own course materials. “It’s 
not surprising, because of lot of publishers have 
someone other than the (textbook) authors come 
up with solution manuals,” says Aaron Weiss, dean 
of science and mathematics. One faculty member 
stopped using adaptive courseware because she 
found it tended to accept some answers she would 
have graded as incorrect, while other questions 
were overly time consuming for students. 
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While many students found adaptive homework 
aligned with what they were doing in class, some 
said the graded work within the assignments often 
felt different or disconnected. “They‘re the same 
questions, but they're worded differently or there’s 
a different step that wasn’t in the lesson you need 
to get it correct,” says Medina. 


Workload. Faculty noted that students spent 
varying amounts of time completing adaptive 
practice assignments. “That’s not necessarily a 
bad thing—the time it takes each student to learn 
something is the time it takes,” Weiss says. “What | 
need to know in the long run is if it’s an appropriate 
amount of time.” 


Some students raised similar concerns, reporting 
that they could spend more than five hours on 
adaptive assignments if they were struggling with 
the content. “It’s a good tool if you're proficient 
and you're trucking along, but if not, it’s very 
different,” says nursing student Sabrina Thompson. 
“Sometimes you get the same questions over and 
over again to boost proficiency, so it doesn’t help in 
the long run.” 


Ultimately, some online homework questions 

were scaled back as a response, Weiss adds, as 
faculty took advantage of the courseware’s ability 
to adjust the length and scope of assignments. 
McFadden made similar adjustments in his business 
courses to ensure that students wouldn‘t spend 
more than 30 to 35 minutes per assignment. “If 

you went by the default values, your students are 
going to be spending a lot of time in some of these 
chapters,” he says. 


Grading. While statistics faculty assigned grades to 
adaptive work to ensure that students consistently 
completed it, several students reported an 
unintended consequence to how adaptive 
courseware selects and grades problems. 


Noting that the courseware identifies strengths 
and weaknesses as students progress through 
adaptive assignments in hopes they'll choose 
additional practice in areas of difficulty, Cristarella 
and other students acknowledge they often 
chose easier sections. “It’s the easiest way to keep 
my grades up,” she says. 
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“I'll use the breakdown to see where I’m weakest,” 
says student Samuel Bitter. “Those are the questions 
I’m most likely to get wrong, but | can choose an 
easier section and keep up my grade. | use it as a 
study tool, but only after | secure my grade.” 


Others pointed out that the courseware can 
reduce student scores if they need multiple 
attempts to solve a problem. “| don’t think it’s 
intentional, but it almost comes across that way,” 
Thompson says. “Those little things could use 
tweaking to not discourage students.” 


Together, these issues also impact how students 
say they perceive adaptive work. “It’s used more 
as an assignment to produce a grade than help,” 
Thompson says. 


English. As with other institutions, faculty struggled 
to find effective courseware for developmental 
English. “Nothing really addressed what we 
needed it to address. Unlike math, right and wrong 
answers are a lot more vague in reading than basic 
math and algebra,” says Dr. Brenda Pongracz, 
dean of arts and humanities. “This would be a really 
good business opportunity for someone.” 


LESSONS LEARNED 


Keys to LCCC’s implementation of adaptive courseware: 


e Focusing on capacity building 
around all kinds of digital learning 
tools. Administrators were initially 
uncertain about participating 
in the Every Learner initiative 
because of ongoing experiments 
with other digital tools, including 
Open Educational Resources 
(OER). Ultimately, they determined 
that experimenting and receiving 
supports and coaching through 
the network would help them 
more broadly build capacity to 
“identify new types of edtech 
and adopt them into our courses,” 
says Karla Aleman, former dean of 
LCCC’s library and eLearning. 


Finding the right faculty 
champions. Implementation in 

all three disciplines began with 

a single faculty member who 
took the lead and supported 

their peers. “There needs to be 
someone with a little bit of prowess 
working with the technology or 
the willingness to investigate it,” 
says Aaron Weiss, dean of science 
and mathematics. “It makes the 
implementation easier.” 


Providing faculty with support for 
implementation—and ongoing 
usage. The Every Learner 

grant helped support faculty 
professional learning and release 
time for implementation efforts, 
according to Weiss. 


Beyond implementation, however, 
it’s still important for administrators 
To acknowledge that additional 


Time will still be required for faculty 
To use adaptive courseware 
effectively, “particularly if it’s being 
used correctly,” Weiss says. 


“You're going in and monitoring 
based on this real-time feedback,” 
he says. “That’s a lot more time 
spent by faculty members.” 


Intentional grading policies. By 
grading adaptive homework, 
statistics faculty ensured that 

the stakes for students were 
higher, but they also made 
efforts fo ensure that the 
threshold to receive full credit 
was lower—/0-80 percent correct 
on adaptive quizzes yields 100 
percent in the course gradebook. 


The importance of supporting 
students. Faculty and students 
stressed the importance of 
providing support outside of the 
courseware (see sidebar, above). 
“You can’t just press play and 
walk away,” says math faculty 
member Kati Dobeck. “You have 
To say ‘you can't settle for this 
grade—you can visit a tutor or use 
these resources.’ That’s absolutely 
an important part of this.” 


“Thinking of adaptive practice 

as a silver bullet to solve all your 
problems is not helpful,” Weiss 
adds. Instead, think of it as “a 
supportive tool designed to be 
used in your course appropriately 
as one more thing to help students 
learn,” he says. 
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Considering implementation a long- 
term effort. Recognizing that faculty 
members made adjustments from 
semester to semester, administrators 
are waiting to see the full impact of 
adaptive courseware. “We need 

to take the long view and allow 
people several semesters to see 
results,” Aleman says. 


Understanding the role of 

faculty in scaling. Efforts to scale 
implementation across multiple 
sections of a course rely heavily on 
how closely faculty and adjuncts 
have historically coordinated, and 
whether faculty members are 
taking the lead. “It is very helpful to 
have someone who says, ‘this is the 
course, this is the content, we‘ve 
done this in one section and we're 
ready to scale,” says Aleman. 
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e Recognizing institutional 


capacity limitations. After 
English faculty members failed 
to find adaptive courseware 
that met their needs in 
developmental courses, vendors 
offered to collaborate ona 
solution—essentially helping 
them build their own product. 
Faculty and administrators 

were cognizant, however, that 
they lacked the bandwidth to 
do so. “It’s not an easy thing, 

or something that’s easy for 
vendors to follow through on. In 
a perfect world, we would give 
faculty time to build something 
online, adaptable, and free— 
like OER,” Weiss says. “Time is the 
hardest part to account for.” 


CONCLUSION 


As faculty in both statistics and business courses 
continue to scale the use of adaptive courseware, 
administrators say that the Every Learner grant has 
opened the door for broader improvement efforts 

at LCCC, including opportunities to collaborate with 
the ATID Network and access to resources including 
coaching, professional development, and networking. 
“From an institutional perspective, if was an opportunity 
to learn from other institutions and get coaching 
around organizational pieces,” says Karla Aleman, 
former dean of LCCC’s library and eLearning. 


Those resources may ultimately help faculty adopt 
adaptive courseware in other math courses and 
disciplines, including science, says Weiss. “The hardest 
part is taking the initial steps and getting someone 
interested — and allowing for the appropriate time for 
the faculty member to research and potentially build 
the tools,” he says. 


For students, the time devoted to implementation to date 
has paid off. As Wade progresses through her second year 
at LCCC, she thinks back to her first math class, saying 

that the adaptive courseware was a significant part of her 
gateway experience. “If was a good fit for me,” she says. 
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